As I've said, being able to convert a paper to code is definitely a hyper power, especially in a field like machine learning which is moving faster and faster each day. Most research papers come from people within giant tech companies or universities who may be PhD holders or the ones who are working on the cutting edge technologies. What else can be more cool than being able to reproduce the research done by these top notch professionals. Another thing to note is that the ones who can reproduce research papers as code is in huge demand. Once you get the knack of implementing research papers, you will be in a state on par with these researchers.

Transformer-XL (meaning extra long) allows for the learning of dependency beyond a fixed-length without disrupting temporal coherence. It incorporates a segment-level recurrence mechanism and a positional encoding scheme. TransformerXL learns dependency that's 80% longer than RNNs and 450% longer than vanilla Transformers. The authors introduce recurrence to their deep self-attention network. Instead of calculating hidden states from scratch for each new segment, they reuse the hidden states obtained in the previous segments.

Today, with contributions made by IBM scientists, IBM introduces Deep Learning as a Service within Watson Studio, a rich set of cloud-based tools for developers and data scientists to help remove the barriers of training deep learning models in the enterprise. Deep learning and machine learning require expensive hardware and software resources as well as more expensive skilled scientists and developers. Deep learning, in particular, requires users to be experts at different levels of the stack, from neural network design to new hardware. Allowing them to be more effective requires cross-stack innovation and software/hardware co-design. The challenges faced in creating AI models and applications have recently been gaining more attention, with Berkeley's Joe Hellerstein highlighting the AI Engineering Gap, a new academic conference (SysML) focused solely on the intersection of systems and machine learning, and the Stanford DAWN initiative calling out the "lack of systems and tools for end to end machine learning development."

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)